fix(proxy): model-level guardrails not executing for non-streaming post_call by michelligabriele · Pull Request #23774 · BerriAI/litellm

michelligabriele · 2026-03-16T19:56:40Z

Model-level guardrails (litellm_params.guardrails on a deployment) were only merged into request metadata in the streaming post_call path (async_post_call_streaming_hook) but not in the non-streaming path (post_call_success_hook). This caused should_run_guardrail to skip the guardrail because the guardrail name was never added to metadata.guardrails.

Add the same _check_and_merge_model_level_guardrails call to post_call_success_hook before the guardrail callback loop, mirroring the streaming path.

Fixes model-level guardrails silently not firing for non-streaming post_call requests.

Relevant issues

Fixes #18363 (partially — the original fix PR #18895 only covered the pre_call path)

Pre-Submission checklist

I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
My PR passes all unit tests on make test-unit
My PR's scope is as isolated as possible, it only solves 1 specific problem
I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix
✅ Test

Changes

Problem

Model-level guardrails configured via litellm_params.guardrails in the config or UI were not being applied for non-streaming post_call requests. The guardrail only fired when explicitly passed in the request body or when using streaming.

Root Cause

_check_and_merge_model_level_guardrails() (which reads deployment.litellm_params.guardrails and merges them into metadata.guardrails) was called in async_post_call_streaming_hook (utils.py:2097) but not in post_call_success_hook (utils.py:1934). Without the merge, should_run_guardrail() found no matching guardrail name in metadata and returned False.

Fix

Added the same _check_and_merge_model_level_guardrails() call to post_call_success_hook before the guardrail callback loop, mirroring the streaming path.

Files Changed

litellm/proxy/utils.py — Added model-level guardrail merge in post_call_success_hook (+5 lines)
tests/test_litellm/proxy/test_model_level_guardrails.py — New test file: 8 unit tests for the merge function + 2 integration tests for the hook

…st_call Model-level guardrails (litellm_params.guardrails on a deployment) were only merged into request metadata in the streaming post_call path (async_post_call_streaming_hook) but not in the non-streaming path (post_call_success_hook). This caused should_run_guardrail to skip the guardrail because the guardrail name was never added to metadata.guardrails. Add the same _check_and_merge_model_level_guardrails call to post_call_success_hook before the guardrail callback loop, mirroring the streaming path. Fixes model-level guardrails silently not firing for non-streaming post_call requests.

vercel · 2026-03-16T19:56:45Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Ready	Preview, Comment	Mar 16, 2026 8:08pm

codspeed-hq · 2026-03-16T19:58:48Z

Merging this PR will not alter performance

✅ 16 untouched benchmarks

_{Comparing michelligabriele:fix/model-level-guardrails-non-streaming-postcall (b0c317e) with main (3dccdde)}

greptile-apps · 2026-03-16T20:00:05Z

Greptile Summary

This PR fixes a bug where model-level guardrails configured via litellm_params.guardrails on a deployment were silently not executing for non-streaming post_call requests, because _check_and_merge_model_level_guardrails() was only called in async_post_call_streaming_hook but not in post_call_success_hook. The fix adds the same merge call to the non-streaming path (5 lines), mirroring the already-working streaming implementation.

Key changes:

litellm/proxy/utils.py: Adds _check_and_merge_model_level_guardrails(data, llm_router) call in post_call_success_hook before the guardrail callback loop; passes the merged guardrail_data to should_run_guardrail() so model-level guardrails are correctly detected.
tests/test_litellm/proxy/test_model_level_guardrails.py: Adds 8 unit tests for the merge helper and 2 async integration tests covering the positive case (model-level guardrail fires) and negative case (unrelated guardrail is skipped).
The fix is correctly scoped and consistent with the streaming path. One minor style improvement would be to guard the _check_and_merge_model_level_guardrails call behind if guardrail_callbacks: to avoid the unnecessary router lookup and dict copy on every request when no guardrail callbacks are registered.

Confidence Score: 4/5

Safe to merge — targeted bug fix with good test coverage and no backwards-incompatible changes.
The core fix is a minimal, correct addition that mirrors an already-working code path. Tests cover the main scenarios using mocks only. The only issue is a minor inefficiency (unconditional router lookup even when no guardrail callbacks exist), which is a style concern rather than a correctness problem.
No files require special attention beyond the minor style note on litellm/proxy/utils.py.

Important Files Changed

Filename	Overview
litellm/proxy/utils.py	Adds `_check_and_merge_model_level_guardrails()` call in `post_call_success_hook` before the guardrail callback loop, mirroring the streaming path. The fix is correct and targeted; a minor inefficiency exists in calling the merge unconditionally even when no guardrail callbacks are registered.
tests/test_litellm/proxy/test_model_level_guardrails.py	New test file with 8 unit tests for `_check_and_merge_model_level_guardrails` and 2 integration tests covering the non-streaming hook path. Tests are mock-only (no real network calls), use `__file__`-relative path resolution, and cover the key positive and negative cases.

Sequence Diagram

sequenceDiagram
    participant Client
    participant ProxyLogging
    participant _check_and_merge
    participant Router
    participant CustomGuardrail

    Client->>ProxyLogging: POST /chat/completions (non-streaming)
    ProxyLogging->>ProxyLogging: Classify callbacks into guardrail_callbacks / other_callbacks
    Note over ProxyLogging: NEW (this PR): merge model-level guardrails
    ProxyLogging->>_check_and_merge: _check_and_merge_model_level_guardrails(data, llm_router)
    _check_and_merge->>Router: get_deployment(model_id)
    Router-->>_check_and_merge: deployment (with litellm_params.guardrails)
    _check_and_merge->>_check_and_merge: merge guardrails into metadata
    _check_and_merge-->>ProxyLogging: guardrail_data (merged)
    loop for each CustomGuardrail
        ProxyLogging->>CustomGuardrail: should_run_guardrail(data=guardrail_data, post_call)
        CustomGuardrail-->>ProxyLogging: True / False
        alt should run
            ProxyLogging->>CustomGuardrail: async_post_call_success_hook(data, response)
            CustomGuardrail-->>ProxyLogging: guardrail_response
        end
    end
    ProxyLogging-->>Client: response

_{Last reviewed commit: b0c317e}

greptile-apps · 2026-03-16T20:00:09Z

tests/test_litellm/proxy/test_model_level_guardrails.py

+import pytest
+from unittest.mock import MagicMock, patch
+
+sys.path.insert(0, os.path.abspath("../../.."))


Fragile relative sys.path manipulation

os.path.abspath("../../..") resolves relative to the process working directory at the time the test is collected, not relative to the test file's location. If pytest is run from any directory other than tests/test_litellm/proxy/, this path will resolve to the wrong location and the subsequent imports will fail or silently import from an unexpected location.

Use __file__-relative path resolution instead, which is stable regardless of the working directory:

Suggested change

sys.path.insert(0, os.path.abspath("../../.."))

sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../..")))

Address Greptile review — resolve sys.path relative to the test file location instead of the process working directory.

greptile-apps · 2026-03-16T20:10:08Z

litellm/proxy/utils.py

+            guardrail_data = _check_and_merge_model_level_guardrails(
+                data=data, llm_router=llm_router


Merge called even when no guardrail callbacks exist

_check_and_merge_model_level_guardrails() is invoked unconditionally after the callback-classification loop, even when guardrail_callbacks is empty (i.e., no CustomGuardrail is registered). This causes an unnecessary llm_router.get_deployment() lookup and dict copy on every non-streaming request, even when there are no guardrail callbacks to trigger.

Guard the call so it only runs when there is at least one guardrail callback to consider:

Suggested change

guardrail_data = _check_and_merge_model_level_guardrails(

data=data, llm_router=llm_router

if guardrail_callbacks:

guardrail_data = _check_and_merge_model_level_guardrails(

data=data, llm_router=llm_router

)

else:

guardrail_data = data

vercel bot deployed to Preview March 16, 2026 19:58 View deployment

greptile-apps bot reviewed Mar 16, 2026

View reviewed changes

test: use __file__-relative sys.path for stable test imports

b0c317e

Address Greptile review — resolve sys.path relative to the test file location instead of the process working directory.

vercel bot deployed to Preview March 16, 2026 20:08 View deployment

greptile-apps bot reviewed Mar 16, 2026

View reviewed changes

krrishdholakia merged commit 0d7425a into BerriAI:main Mar 18, 2026
36 of 39 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix(proxy): model-level guardrails not executing for non-streaming post_call#23774

fix(proxy): model-level guardrails not executing for non-streaming post_call#23774
krrishdholakia merged 2 commits intoBerriAI:mainfrom
michelligabriele:fix/model-level-guardrails-non-streaming-postcall

michelligabriele commented Mar 16, 2026

Uh oh!

vercel bot commented Mar 16, 2026 •

edited

Loading

Uh oh!

codspeed-hq bot commented Mar 16, 2026 •

edited

Loading

Uh oh!

greptile-apps bot commented Mar 16, 2026 •

edited

Loading

Important Files Changed

Uh oh!

greptile-apps bot Mar 16, 2026

Uh oh!

greptile-apps bot Mar 16, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	sys.path.insert(0, os.path.abspath("../../.."))
	sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), "../../..")))

		guardrail_data = _check_and_merge_model_level_guardrails(
		data=data, llm_router=llm_router

Uh oh!

Conversation

michelligabriele commented Mar 16, 2026

Relevant issues

Pre-Submission checklist

Type

Changes

Problem

Root Cause

Fix

Files Changed

Uh oh!

vercel bot commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codspeed-hq bot commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

greptile-apps bot commented Mar 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Mar 16, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

vercel bot commented Mar 16, 2026 •

edited

Loading

codspeed-hq bot commented Mar 16, 2026 •

edited

Loading

greptile-apps bot commented Mar 16, 2026 •

edited

Loading